Magnetic-Visual Sensor Fusion-based Dense 3D Reconstruction and Localization for Endoscopic Capsule Robots

Reliable and real-time 3D reconstruction and localization functionality is a crucial prerequisite for the navigation of actively controlled capsule endoscopic robots as an emerging, minimally invasive diagnostic and therapeutic technology for use in the gastrointestinal (GI) tract. In this study, we propose a fully dense, non-rigidly deformable, strictly real-time, intraoperative map fusion approach for actively controlled endoscopic capsule robot applications which combines magnetic and vision-based localization, with non-rigid deformations based frame-to-model map fusion. The performance of the proposed method is demonstrated using four different ex-vivo porcine stomach models. Across different trajectories of varying speed and complexity, and four different endoscopic cameras, the root mean square surface reconstruction errors 1.58 to 2.17 cm.Comment: submitted to IROS 201

Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robots

In the last decade, many medical companies and research groups have tried to convert passive capsule endoscopes as an emerging and minimally invasive diagnostic technology into actively steerable endoscopic capsule robots which will provide more intuitive disease detection, targeted drug delivery and biopsy-like operations in the gastrointestinal(GI) tract. In this study, we introduce a fully unsupervised, real-time odometry and depth learner for monocular endoscopic capsule robots. We establish the supervision by warping view sequences and assigning the re-projection minimization to the loss function, which we adopt in multi-view pose estimation and single-view depth estimation network. Detailed quantitative and qualitative analyses of the proposed framework performed on non-rigidly deformable ex-vivo porcine stomach datasets proves the effectiveness of the method in terms of motion estimation and depth recovery.Comment: submitted to IROS 201

Electrokinetic confinement of axonal growth for dynamically configurable neural networks

Axons in the developing nervous system are directed via guidance cues, whose expression varies both spatially and temporally, to create functional neural circuits. Existing methods to create patterns of neural connectivity in vitro use only static geometries, and are unable to dynamically alter the guidance cues imparted on the cells. We introduce the use of AC electrokinetics to dynamically control axonal growth in cultured rat hippocampal neurons. We find that the application of modest voltages at frequencies on the order of 10[superscript 5] Hz can cause developing axons to be stopped adjacent to the electrodes while axons away from the electric fields exhibit uninhibited growth. By switching electrodes on or off, we can reversibly inhibit or permit axon passage across the electrodes. Our models suggest that dielectrophoresis is the causative AC electrokinetic effect. We make use of our dynamic control over axon elongation to create an axon-diode via an axon-lock system that consists of a pair of electrode ‘gates’ that either permit or prevent axons from passing through. Finally, we developed a neural circuit consisting of three populations of neurons, separated by three axon-locks to demonstrate the assembly of a functional, engineered neural network. Action potential recordings demonstrate that the AC electrokinetic effect does not harm axons, and Ca[superscript 2+] imaging demonstrated the unidirectional nature of the synaptic connections. AC electrokinetic confinement of axonal growth has potential for creating configurable, directional neural networks.National Institutes of Health (U.S.) (R01 EUREKA Award R01-NS066352

Ultra-rapid laser protein micropatterning: screening for directed polarization of single neurons

Protein micropatterning is a powerful tool for studying the effects of extracellular signals on cell development and regeneration. Laser micropatterning of proteins is the most flexible method for patterning many different geometries, protein densities, and concentration gradients. Despite these advantages, laser micropatterning remains prohibitively slow for most applications. Here, we take advantage of the rapid multi-photon induced photobleaching of fluorophores to generate sub-micron resolution patterns of full-length proteins on polymer monolayers, with sub-microsecond exposure times, i.e. one to five orders of magnitude faster than all previous laser micropatterning methods. We screened a range of different PEG monolayer coupling chemistries, chain-lengths and functional caps, and found that long-chain acrylated PEG monolayers are effective at resisting non-specific protein adhesion, while permitting efficient cross-linking of biotin-4-fluorescein to the PEG monolayers upon exposure to femtosecond laser pulses. We find evidence that the dominant photopatterning chemistry switches from a two-photon process to three- and four-photon absorption processes as the laser intensity increases, generating increasingly volatile excited triplet-state fluorophores, leading to faster patterning. Using this technology, we were able to generate over a hundred thousand protein patterns with varying geometries and protein densities to direct the polarization of hippocampal neurons with single-cell precision. We found that certain arrays of patterned triangles as small as neurite growth cones can direct polarization by impeding the elongation of reverse-projecting neurites, while permitting elongation of forward-projecting neurites. The ability to rapidly generate and screen such protein micropatterns can enable discovery of conditions necessary to create in vitro neural networks with single-neuron precision for basic discovery, drug screening, as well as for tissue scaffolding in therapeutics.Hertz Foundation (Fellowship)National Institutes of Health (U.S.) (R01 EUREKA Award 1-R01-NS066352)David & Lucile Packard Foundation (Award in Science and Engineering

Noninvasive electron microscopy with interaction-free quantum measurements

We propose the use of interaction-free quantum measurements with electrons to eliminate sample damage in electron microscopy. This might allow noninvasive molecular-resolution imaging. We show the possibility of such measurements in the presence of experimentally measured quantum decoherence rates and using a scheme based on existing charged particle trapping techniques.David and Lucile Packard Foundatio

Fully automated cellular-resolution vertebrate screening platform with parallel animal processing

The zebrafish larva is an optically-transparent vertebrate model with complex organs that is widely used to study genetics, developmental biology, and to model various human diseases. In this article, we present a set of novel technologies that significantly increase the throughput and capabilities of our previously described vertebrate automated screening technology (VAST). We developed a robust multi-thread system that can simultaneously process multiple animals. System throughput is limited only by the image acquisition speed rather than by the fluidic or mechanical processes. We developed image recognition algorithms that fully automate manipulation of animals, including orienting and positioning regions of interest within the microscope's field of view. We also identified the optimal capillary materials for high-resolution, distortion-free, low-background imaging of zebrafish larvae.National Institutes of Health (U.S.) (Director's New Innovator Award DP2 OD002989)National Institutes of Health (U.S.) (Transformative Research Award R01 NS073127)David & Lucile Packard Foundation (Award in Science and Engineering)Broad Institute of MIT and Harvard (SPARC Award)Foxconn International Holdings Ltd.Athinoula A. Martinos Center for Biomedical Imaging (Training Grant

Submicrometer All-Optical Digital Memory and Integration of Nanoscale Photonic Devices Without Isolators

Abstract-In this paper, the authors introduce multibit all-optical memory devices in nanostructured photonic-crystal circuits using only intrinsic nonresonant optical nonlinearities of semiconductors. Introduced devices can record incoming pulses at speeds of 10 Gb/s using power levels less than 1 mW or at speeds approaching 70 Gb/s using power levels of 10 mW. The incoming pulses are recorded in high-contrast digital output levels independent of the input bit format. The devices exhibit tunable gain for fan-out with negligible reflection and low dissipation and can provide signal regeneration, including reshaping and retiming. Separate signal, clock and reset inputs, and memory outputs coexist without any crosstalk. Input, clock, and output operating frequencies can be independently tuned. By simulating the operation of such all-optical memory devices, it is also shown that nanoscale optical devices can be cascaded to construct densely integrated systems without any isolators or amplifiers, even in the presence of reflections

Machine learning reveals interhemispheric somatosensory coherence as indicator of anesthetic depth

The goal of this study was to identify features in mouse electrocorticogram recordings that indicate the depth of anesthesia as approximated by the administered anesthetic dosage. Anesthetic depth in laboratory animals must be precisely monitored and controlled. However, for the most common lab species (mice) few indicators useful for monitoring anesthetic depth have been established. We used electrocorticogram recordings in mice, coupled with peripheral stimulation, in order to identify features of brain activity modulated by isoflurane anesthesia and explored their usefulness in monitoring anesthetic depth through machine learning techniques. Using a gradient boosting regressor framework we identified interhemispheric somatosensory coherence as the most informative and reliable electrocorticogram feature for determining anesthetic depth, yielding good generalization and performance over many subjects. Knowing that interhemispheric somatosensory coherence indicates the effectively administered isoflurane concentration is an important step for establishing better anesthetic monitoring protocols and closed-loop systems for animal surgeries

High-throughput hyperdimensional vertebrate phenotyping

Most gene mutations and biologically active molecules cause complex responses in animals that cannot be predicted by cell culture models. Yet animal studies remain too slow and their analyses are often limited to only a few readouts. Here we demonstrate high-throughput optical projection tomography with micrometer resolution and hyperdimensional screening of entire vertebrates in tens of seconds using a simple fluidic system. Hundreds of independent morphological features and complex phenotypes are automatically captured in three dimensions with unprecedented speed and detail in semi-transparent zebrafish larvae. By clustering quantitative phenotypic signatures, we can detect and classify even subtle alterations in many biological processes simultaneously. We term our approach hyperdimensional in vivo phenotyping (HIP). To illustrate the power of HIP, we have analyzed the effects of several classes of teratogens on cartilage formation using 200 independent morphological measurements and identified similarities and differences that correlate well with their known mechanisms of actions in mammals

High-throughput hyperdimensional vertebrate phenotyping

Most gene mutations and biologically active molecules cause complex responses in animals that cannot be predicted by cell culture models. Yet animal studies remain too slow and their analyses are often limited to only a few readouts. Here we demonstrate high-throughput optical projection tomography with micrometre resolution and hyperdimensional screening of entire vertebrates in tens of seconds using a simple fluidic system. Hundreds of independent morphological features and complex phenotypes are automatically captured in three dimensions with unprecedented speed and detail in semitransparent zebrafish larvae. By clustering quantitative phenotypic signatures, we can detect and classify even subtle alterations in many biological processes simultaneously. We term our approach hyperdimensional in vivo phenotyping. To illustrate the power of hyperdimensional in vivo phenotyping, we have analysed the effects of several classes of teratogens on cartilage formation using 200 independent morphological measurements, and identified similarities and differences that correlate well with their known mechanisms of actions in mammals.National Institutes of Health (U.S.) (NIH Transformative Research Award (R01 NS073127))National Institutes of Health (U.S.) (NIH (R01 GM095672)National Institutes of Health (U.S.) (NIH Director’s New Innovator award (1-DP2-OD002989))Howard Hughes Medical Institute (International Student Fellowship)Broad Institute of MIT and Harvard (SPARC grant)David & Lucile Packard Foundation (Award in Science and Engineering